Learning-Based Facial Animation

This thesis proposes a novel approach for automated 3D speech animation from audio. An end-to-end system is presented which undergoes three principal phases. In the acquisition phase, dynamic articulation motions are recorded and amended. The learning phase studies the correlation of these motions in their phonetic context in order to understand the visual nature of speech. Finally, for the synthesis phase, an algorithm is proposed that carries as much of the natural behavior as possible from the acquired data to the final animation. The selection of motion segments for the synthesis of animations relies on a novel similarity measure, based on a Locally Linear Embedding representation of visemes, which closely relates to viseme categories defined in articulatory phonetics literature. This measure offers a relaxed selection of visemes, without reducing the quality of the animation. Along with a general hierarchical substitution procedure which can directly be reused in other speech animation systems, our algorithm performs optimum segment concatenation in order to create new utterances with natural coarticulation effects.